1. Field of Invention
The present invention relates to movie display systems, and more specifically, to interactive implementations that provide for dynamic selection of displayed movie segments and control of the temporal presentation rate of these segments.
2. Discussion of Prior Art
xe2x80x9cMotion Picturexe2x80x9d or video imagery in the form of a sequence of still frame images presented in rapid successionxe2x80x94a xe2x80x9cmoviexe2x80x9dxe2x80x94is a commonly understood effective and powerful method of conveying dynamically changing images to humans.
Movies are today used in some manner in virtually all industry and consumer contexts. Applications for movies include entertainment, advertising, group presentation, education, assembly and repair, art, scientific visualization, accident scene re-enactment, and many others.
Note that by xe2x80x9cmoviexe2x80x9d, we refer to the presentation of images sequences slightly differing in appearance such that the appearance of motion (or time evolution of another parameter, such as scene lighting, if object position is not the time-evolving element) is perceived by viewers. We do not require that a xe2x80x9cmoviexe2x80x9d be of particular length, stored on a particular media, have a particular frame rate, etc. Thus we include in the content realm TV shows, advertisements, computer-generated scientific visualizations, etc., as well as full length feature Hollywood films. For storage media we include videotape, film, Compact Disk (CD), Digital Video Disk (DVD), computer hard disk, etc. Further, we use the word xe2x80x9cmoviexe2x80x9d to include image sequences or image and audio sequences combined.
In consequence of movies"" effectiveness in many applications, not only is the use of movies accepted and widespread, but movies are already available containing images on virtually any subject one could imagine. Such movie images come both from the physical world and from computer graphics. Furthermore, inventions such as consumer camcorders, and to a somewhat lesser extent general-purpose computer animation packages, make creating new movie image content a relatively inexpensive and straightforward matter even for the individual consumer.
While movies are a powerful communications medium, in their common viewing modes they suffer from a number of disadvantages that manifest themselves in many important presentation contexts. These include:
1) Requirement of Audience Attention for Significant Time Periods: For effective presentation, movies typically require audience attention over an extended period of time. In locations such as trade shows, science museums, retail environments, and other similarly time-limited viewing contexts, this can be a significant problem.
2) Lack of Novelty: Because of the proliferation of movies throughout all aspects of our society via TV and other media, movies as a presentation medium lacks novelty for many viewers. In order for movie-based displays to get viewers"" attention, therefore, they can require outstanding, and thus typically very expensive, image content and organization production work in the form of filming and editing, computer graphics, or both.
3) Difficulty to Quickly Locate Information of Interest: In environments in which losing the attention of the audience is not the main problem (such as research and video editing environments), it is nevertheless the case that on movie segments, particularly lengthy ones, there can be significant difficult finding particular information in a timely and efficient manner.
4) Lack of Ability to Adequately Dynamically Alter the Rate and Manner of Movie Presentation:
[xe2x80x9cMannerxe2x80x9d is included in the phrase xe2x80x9crate and manner of movie presentationxe2x80x9d to convey is the idea that the frame progression may be a complex highly non-linear function of time. For example, consider a film of a person walking across the floor of a room at a uniform rate. Imagine then that the time advance of the resulting movie followed the equation:
FrameNumber=A+Bsin(Ct) where xe2x80x9cAxe2x80x9d, xe2x80x9cBxe2x80x9d and xe2x80x9cCxe2x80x9d are appropriately chosen constants and xe2x80x9ctxe2x80x9d is the actual (viewer-perceived) time. In such an instance (assuming uniform temporal frame spacing), the person would seem to be walking backwards and forwards, pausing at each end of the room.]
andandandand In instances where time compression, dilation, or mathematical functional modification of the presentation rate of movie content is entertaining, useful in apprehending and understanding the movie content, or educational as regards comprehending the flow of time, available viewing environments lack one or more desirable features. Such desirable features include:
a) The ability to view clear, smooth non-distorted images all the way from single-frame stills, to motion sequences running potentially arbitrarily (tens, hundreds, thousands, etc. times) faster (in the plus or minus sense) than their standard presentation rate. Note that by xe2x80x9csmoothxe2x80x9d, what is meant is the lack of a (typically) brief but nevertheless intrusive freezing of the desired time progression of viewed movie image frames, due to (computer) disk access, data transfer, frame decompression time, or other such reasons.
Note that while rate-altered capture schemes and non-real-time movie regeneration methods have important uses, they do not substitute for dynamic alteration of the movie""s presentation rate interactively at view time by user manipulated controls, user choice of mathematical function, music, environmental parameters (such as light-level, etc.), or other view-time-dependent input.
b) The ability to alter the frame index, frame rate, or more complex mathematical function determining frame progression, conveniently and naturallyxe2x80x94ideally interactivelyxe2x80x94as the movie is being viewed. Input specifications for time-evolution may include:
i) Rotary dials attached to (optical or other) shaft encoders whose position determines the absolute time within the movie, the rate of time progression through the movie, or other more complex viewed-frame specification methods. Such dials could optionally include a detent that engages the standard 24-30 fps playback rate, optionally with audio.
ii) Algorithms that take music signals as inputs and generate time flows as outputs, in order to enable movie imagery to automatically xe2x80x9cdancexe2x80x9d to music.
iii) Pressure-sensing xe2x80x9cdrumxe2x80x9d pads, foot-pedals, or the like, that in conjunction with (software) algorithms generate time flows as output, whereby performers may xe2x80x9cplayxe2x80x9d movies (of images such as, for example, the rising of the sun) like a visual analog of a musical instrument.
iv) Other electro/optical-mechanical user actuator, environmental sensor, or algorithmic mechanisms and methods.
c) A modest system cost, ruggedness, and high level of user-friendliness.
Attempts have been made to build movie presentation devices that address one or more of these limitations. These include:
1) xe2x80x9cMultimediaxe2x80x9d Systems: (Such systems attempt to address the issues of Requirement of Audience Attention for Significant Time Periods and Difficulty to Quickly Locate Information of Interest.): In such environments movie information is typically stored in a xe2x80x9ctreexe2x80x9d structured fashion on a random access media such as a Compact Disk Read Only Memory (CD ROM). While helping to reduce the time for a viewer to locate a movie segment-start, this solution has a number of significant disadvantages. These include one or more of:
a) Expensive production costs to design and implement such a tree-structured software environment for each presentation having new movie image content.
b) Necessity of knowing a-priori the movie indexing parameters of interest to final viewers. (Attempts have been made to create computer-automated indexing schemes. However these solutions have the drawback that they are typically extremely complex and expensive to implement, and each new indexing parameter typically requires a new indexing algorithm to be developed.)
c) Reliance on common navigation mechanisms (computer mice, touch screens, etc.) used in conventional ways (such as dragging a slider along a short linear bar) that are neither particularly suited to the application in question nor engaging for the average viewer.
d) Inability to easily and naturally control the advance through movie image sequences over wide ranges of rates in a manner determined dynamically by the user through one or more of rotary dials, mathematical algorithms, environmental parameters, music, or other input signal or mechanical control.
e) Inability to present visually clear images and temporally smooth frame transitions at all viewing rates over the entire length of the movie.
f) Highly specialized graphical user interfaces ill suited for frequently updating the display inexpensively with new movies showing widely differing content.
2) Very Large Movie Images, 3-D Movie Images Requiring Special Glasses, Moving Movie-Theater Seats, Etc.: (Such schemes attempt to address the issue of Lack of Novelty.): These and other similar prior specialty movie making, projecting, and viewing methods have a number of significant drawbacks including one or more of:
a) High cost to produce.
b) Projection mechanism complex, expensive, and hard to transport.
c) Inappropriate for space-limited presentation applications.
d) Lack of audience interactivity.
e) Inability to easily and naturally control the advance through movie image sequences, over wide ranges of rates and in a manner determined dynamically by the user through one or more of rotary dials, mathematical algorithms, environmental parameters, music, or other input signals or mechanical controls.
3) DVD Movie Players, Conventional General Purpose Computer (i.e., PC, MAC, or the equivalent)-based Multimedia Movie Players such as Apple Corporation""s Quicktime Software, and Conventional Video Tape Players: (Such systems make efforts to address the issues of Difficulty to Quickly Locate Information of Interest and Lack of Ability to Adequately Dynamically Alter the Rate and Manner of Movie Presentation.): These systems have a number of significant drawbacks including one or more of:
a) Very limited control of the rate and manner of time evolution of the viewed movie.
i) Typically xe2x80x9cfast-forward/reversexe2x80x9d rates are very limited compared to the present movie display invention. Attempting to exceed these very limited rates offered by existing systems (if possible within fast-forward/reverse modes, or if not, by dragging an on-screen slider, multiply pressing a button an a remote control unit, etc.) typically results in visual frame degradation, temporal inter-frame xe2x80x9cglitchingxe2x80x9d, or both.
ii) User interfaces ill-suited for delicate and flexible interactive control of a movies time evolution.
iii) Inability to control the time evolution of movie frame presentation in any sophisticated manner (such as, for example, from beat information extracted from algorithmically-processed music).
b) Ill-designed for presentation and public installation purposes: Problems typically include one or more of:
i) Undesirable visual screen clutter (e.g., icons, menus, etc.).
ii) Lack of desired controls and features (e.g., a table-of-contents button).
iii) Inability to remove controls that allow the public inappropriate access to the system (e.g., the ability to exit the program, delete a movie, etc.)
iv) Inadequately rugged.
4) Semiconductor Random-Access-Memory (RAM) based perusal systems. (Such systems, in certain select instances, make efforts to address the issues of Difficulty to Quickly Locate Information of Interest, Lack of Ability to Adequately Dynamically Alter the Rate and Manner of Movie Presentation, and Lack of Novelty.):
These systems suffer from the severe drawback that to view clear, smooth images in the desired temporal glitch-free manner, movie content must fit entirely in RAM. Even using image compression, this so limits the amount of movie content as to preclude use in many important applications.
In accordance with the present movie display invention, a system for entertainment, advertising, group presentation, education, assembly and repair, art, scientific visualization, accident scene re-enactment, research, direction-providing, video-editing assistance, gaming, and animation comprised of:
storing movie imagery in a substantially random access media;
connecting to this substantially random access media an image output device for the movie imagery;
providing a means for transferring spontaneously-chosen arbitrary sequences of movie frames from the mass storage device to the display medium while preserving image quality and temporal fluidity;
providing a means for users at view time to dynamically control the progression through stored movie imagery, and thus control the effective flow of time, via one or more of: rotary dials, music analysis algorithms, mathematical functions, or other mechanical, environmental stimuli, or algorithmic means.
Accordingly, several objects and advantages of the present movie display invention are:
1) To provide a method and apparatus for convenient control of frame advance within a movie, at view time, through one or more of rotary dials, music analysis algorithms, mathematical functions, or other user-controlled, environmental stimuli, or algorithmic methods.
[Note that in the case of rotary dial use, more than one rotary dial may be used, where each dial has a different frame-traverse sensitivity, in order that time scales from pico-seconds to millennia may be comfortably traversed. Similarly, some dials may control the position in the frame sequence, others the rate of progression through the frame sequence, and yet others implementing more complex frame advance algorithms. Such algorithms could include frame advance sensitivity that is itself a complex function of dial position, rotation rate, acceleration, etc. Furthermore, such dials could optionally include a detent that engages the standard 24-30 fps playback rate, optionally with audio.]
2) To provide a method and apparatus for viewing, clearly and smoothly, movies presented at highly variable rates of progression (still-frame to potentially tens, hundreds, thousands, etc. times faster than normal viewing rates) over the entire length of the movie. (In most circumstances audio would be disabled or reduced in volume during high-speed traverse, but may be included if desired.)
3) To provide a method and apparatus for implementing interactive movie-based signs that are dynamic, highly flexible in appearance, and eye-catching.
4) To provide a high movie-viewing novelty factor for entertaining and attracting diverse audiences to already existing or newly created movies for the purposes of entertainment, advertising, education, and other applications benefiting from an ability to attract audiences.
5) To provide a mechanism for existing (and specially created) movie sequences to dance to music.
6) To provide an effective way of presenting movie information to audiences having limited available viewing time or short attention spans.
7) To provide an effective way for researchers, tourists, students, repair personnel, and others to locate, review, and peruse information of interest within movie segments.
8) To provide an effective method and apparatus for apprehending and understanding movie contents, particularly as such contents relate to time-evolving processes. Such contents could include movie images of chaotic and non-chaotic physical processes (vibrations, phase transitions, many-particle equilibrium states, crystal formation, biological growth, etc.), sporting events, trips from one location to another for the purpose of providing directions, traffic patterns, weather, dance performances, automated manufacture of consumer goods, graphic evolving artworks, and many other possibilities too extensive to exhaustively enumerate.
9) To provide a way to educate viewers generally as regards comprehending the flow of time.
10) To provide a tool by which existing movie compositions may be conveniently explored in order to better understand movie producers"" stylistic and editing choices from an artistic perspective.
11) To provide a method and apparatus by which it can be made to appear, through a shared functional dependence on time, that users have control of a physical process or action occurring within a movie scene.
For example, consider first filming a room being lit progressively more brightly as the result of someone turning a dimmer at a temporally uniform rate. Then, when such a movie is presented with (for example) a rotary dial frame-advance control, it will appear to the user that they are controlling the light dimmer mechanism itself A similar effect would result if one were to present a gear train, factory automation mechanism, or the like, filmed moving at a uniform rate. Indeed, by choosing the effective time-flow (i.e., frame-advance) sensitivity of the physical rotary dial appropriately, one could make the physical dial turn at exactly the same rate as a corresponding visual element in the movie scene, thus increasing the strength of this xe2x80x9cillusionxe2x80x9d even further.
And finally, by making a physical actuator appear to be a continuation of an element in the movie scene, such an illusion is enhanced even further.
Many other applications of this general principle exist that are too extensive to enumerate.
Note that a key feature of this system, distinguishing it strongly from video games, virtual reality systems, computer simulations, and the like, is that the described user experience of scene control in each specific instance may be achieved with virtually no custom programming.
12) To provide a method and apparatus with the above desirable features where in addition:
a) Movie frame sequences of differing temporal densities may be conveniently saved without significant waste of storage media space.
In this way, processes occurring on times scales all the way from pico-seconds to millennia may be easily stored and traversed. Whereby audience may explore processes ranging from such events as atomic electron orbit energy transitions to continental drift, from nerve impulses travelling along a sprinter""s legs to the lifetime aging of the sprinter. (For reasons of both practicality and real-world physical limitations, movies portraying such tiny or huge time scales would typically be created by computer graphic means.)
b) Each movie frame may be individually compressed in order to save storage media space.
c) Movie images are free from overly numerous menu buttons or other undesirable images.
d) Text and graphic annotation and progress bar image(s) appear at view time as determined by the location currently being viewed in the movie. (The progress bar may include a numeric as well as graphic time-within-movie indicator. It may also include a sliding alphabetized list of movie segment names.)
e) The display, after a predetermined period of no user activity, enters an xe2x80x9cattractxe2x80x9d mode, such that users are drawn to the display when it is unattended. This attract mode optionally includes such features as:
i) Moving from one stored movie segment to another, in order that each of the stored movies is guaranteed to be seen by audiences. Such transitions could occur via sophisticated temporal paths that result in movie scene evolutions that are both complex and alluring.
ii) If a dial actuator is used:
(1) Having the physical dial turned by means of a small motor. (The movie could advance in response to such turning or not as is desired.)
(2) Having a tiny movie image of the physical display itself, where the tiny dial turns as the associated tiny movie advances.
iii) Having musical songs play intermittently, where the movies"" effective advance time is controlled by algorithms applied to these songs, in order to have the movie imagery appear to xe2x80x9cdancexe2x80x9d to the song.
f) There exists a simple user interface such that the display can be used by typical members of the public with no prior training.
g) Conventional movies may be converted the required display format simply and quickly.
h) The display is rugged, user-friendly, and of modest system cost.
i) The user may, by pressing a momentary pushbutton or other similar activation means, initiate a movie image sequence to play forward at a predetermined rate (which could be 0, the normal 30 frame/sec rate, or some other appropriate predetermined rate) along with associated audio.
Such an automated visual and audio playback sequence may start at the current frame being viewed, or may automatically be reset to begin at the start of some predetermined sectionxe2x80x94typically the section from which the currently viewed frame originates.
The accompanying audio may be tightly time-synchronized to the images being viewed, as would typically be the case when the audio is intrinsically connected to the displaying imagesxe2x80x94for example, a movie showing a singer in performance. Alternatively, the audio may be a narrative unconnected with the intrinsic time evolution of the eventxe2x80x94for example, a narrative accompanying images of the blooming of a rose.
j) Advertisers, educators, and others may track the fraction of time that a given movie segment is chosen for viewing by audiences. In this way, advertisers may be billed an amount proportionate to their product""s or service""s exposure, educators may learn which presented subject areas are particularly interesting to, or particularly difficult for, various audiences, etc.
k) Movie frame advance will pause for a predetermined amount of time at a movie boundary, in order that high-speed (typically dial-actuated) traversals do not xe2x80x9cfly pastxe2x80x9d short movie segments.
l) Users may, by pressing a momentary pushbutton or other similar activation control, cause the above-described display to move to a particular frame within its stored sequence.
m) Users depressing a xe2x80x9cTable-of-Contentsxe2x80x9d button will cause:
i) Display on the viewing screen of a table of contents consisting of names and/or descriptions of the stored movies.
ii) An indicatorxe2x80x94for example an arrowxe2x80x94to be present showing where in this movie list the current movie frame display pointer lies (that is, where the user""s viewing position is within the stored movie(s)).
iii) Turns of the movie control dial, or other such control mechanism, to move this arrow through the table of contents list. The effect of this is such that when the Table-of-Contents button is released, the movie frame display pointer (and thus the user""s position within the stored movie(s)) will be changed to display the new location within (a potentially different) movie segment as determined by the table of contents arrow.
Still further objects and advantages will become apparent from consideration of the ensuing description and drawings.